\documentclass{article}
\usepackage{fullpage}
\usepackage{bigbox}
\usepackage{verbatim}

\begin{document}

\title{Creating and Evaluating Surrogate Models\\with Dakota and Surfpack}
\author{Dakota Development Team}
\date{\today}

\maketitle

This note offers a brief summary of ways to build and evaluate a
Gaussian process surrogate model with both Dakota and Surfpack,
including integrating the surrogate evaluation into another
application via a C API.  The discussion and examples here apply not
only to Gaussian process models, but to the following Dakota global
surrogate types, all provided by the Surfpack sub-package:
\begin{itemize}
\item {\tt gaussian\_process surfpack} ({\tt kriging surfpack})
\item {\tt mars} 
\item {\tt moving\_least\_squares}
\item {\tt neural\_network}
\item {\tt radial\_basis}
\item {\tt polynomial}
\end{itemize}
The Dakota-only approaches described will also work for other Dakota
surrogate models, but the Surfpack-specific approaches will not.  All
examples here use a two variable form of the textbook example function
provided with Dakota, so are creating a surrogate model for the
textbook function $f(x1, x2)$.

{\bf Requirements:} The Dakota option to save a Surfpack model file
and some of the standalone Surfpack capabilities require Dakota 5.4 or
later.  Saving (from Dakota or Surfpack) and later loading (and
subsequently evaluating) surrogate models with the Surfpack executable
require the Surfpack standalone and Boost serialization options to be
enabled at compile time.  Building and evaluating surrogate models
directly in Dakota has no special requirements.

The example described here can be found in {\tt
  dakota/examples/eval\_surrogate}.  It contains the following input
files:\\
\begin{bigbox}
\begin{verbatim}
build_points.dat        Training data used to build the surrogate model
eval_points.dat         Points at which to query the surrogate model
dakota_surrogate.in     Dakota input to build/save/evaluate a surrogate 
sp_build_surrogate.spk  Surfpack commands to build/save a surrogate
sp_eval_surrogate.spk   Surfpack commands to load/evaluate a surrogate
\end{verbatim}
\end{bigbox}
And the following example output files:\\
\begin{bigbox}
\begin{verbatim}
dak_gp_model.textbook.sps  Surfpack model file generated by Dakota
dak_surrogate_evals.dat    Dakota-generated evals of the surrogate model
sp_gp_model.textbook.sps   Surfpack model file generated by Surfpack
sp_surrogate_evals.dat     Surfpack-generated evals of the surrogate model
\end{verbatim}
\end{bigbox}

\section{Building a Surrogate Model}

\subsection{Using Dakota}

The examples B1 and B2 presented in this section demonstrate two ways
to build a surrogate model, solely using Dakota.  Later, examples E1
and E2 demonstrate evaluating the previously built models using
Dakota.  The Dakota build and evaluation examples are all based on
running {\tt dakota -input dakota\_surrogate.in}.  This input file is
shown here.

\hrulefill
\verbatiminput{dakota_surrogate.in}
\hrulefill

\subsubsection*{B1: Dakota-generated points design}

Build data option B1 uses the LHS method in Dakota to generate the x1,
x2 points at which to build the surrogate model.  Dakota will generate
10 points at which to build the surrogate model and run the simulation
interface at each point to get the response function value.  It will
then construct the surrogate model and write it to a Surfpack
formatted model file {\tt dak\_gp\_model.textbook.sps}.

\subsubsection*{B2: User-provided points file}

With build data option B2, Dakota accepts a user-provided data file
(here {\tt build\_points.dat}) in freeform data format (whitespace
separated data with rows containing [x1, x2, function]).  The
surrogate model is constructed from these data, and again written to a
Surfpack formatted model file {\tt dak\_gp\_model.textbook.sps}.

Contents of the build data file {\tt build\_points.dat}:\\
\begin{bigbox}
\verbatiminput{build_points.dat}
\end{bigbox}

Dakota also supports an annotated input data file format with a header
row of labels and a leading column with point numbers.  For this case
the number of DesignMethod {\tt samples = 0} in the Dakota input file,
indicating that Dakota should not generate any additional DOE points
beyond those provided in the user data file.

\subsection{Using the Surfpack Executable}

To build the same surrogate model using the standalone Surfpack
executable {\tt surfpack}, use the Surfpack instruction file {\tt
  sp\_build\_surrogate.spk}, together with the data file 
{\tt build\_points.dat}:\\
\begin{bigbox}
\verbatiminput{sp_build_surrogate.spk}
\end{bigbox}

To build, run {\tt surfpack sp\_build\_surrogate.spk}, which will
write the file {\tt sp\_gp\_model.textbook.sps}.  In this example the
Surfpack-generated GP model will differ slightly from that generated
by Dakota.  This is being investigated.

\section{Evaluating a Surrogate Model}

This section discusses evaluating a surrogate model either in Dakota
directly or in Surfpack after previously saving in Dakota or Surfpack.

\subsection{Using Dakota}

The examples in this section use the Dakota input file {\tt
  dakota\_surrogate.in} provided above.  Dakota can use a constructed
surrogate model in any of its iterative methods, for example to
perform optimization on the surrogate model to find a best design.
The simplest methods are to evaluate the surrogate at a user-specified
set of points and to randomly sample the surrogate with a Monte Carlo
method.  The Dakota input file shown above demonstrates both.

To evaluate the surrogate model repeatedly at different sets of points
without rerunning the expensive simulation, change the evaluation
method specification (E1 or E2) and use Dakota's restart facility,
e.g., {\tt dakota -input dakota\_surrogate.in -read\_restart
  dakota.rst}

\subsubsection*{E1: User-provided list of points}

The first example evaluates the surrogate model at the list of points
provided in the Dakota input file.  Dakota writes the surrogate
evaluations to a file {\tt dak\_surrogate\_evals.dat}:\\
\begin{bigbox}
\begin{verbatim}
%eval_id             x1             x2       textbook 
       1            0.9              1 5.695149128e-05 
       2           0.95              1 9.909938326e-06 
       3              1              1 -1.681528987e-06 
       4              1           0.93 3.257983696e-07 
       5              1           0.98 -2.025243345e-07 
       6              1           1.01 -3.01510746e-06 
\end{verbatim}
\end{bigbox}

\subsubsection*{E2: Dakota-generated points design}

In this example, Dakota generates a 100 sample LHS design and evaluates
the surrogate model at the generated points.  Dakota writes the
surrogate evaluations to a file {\tt dak\_surrogate\_evals.dat}:\\
\begin{bigbox}
\begin{verbatim}
%eval_id             x1             x2       textbook 
       1    1.035667472   0.9359510107 2.782730535e-06 
       2   0.9733842609   0.9604024011 5.039401052e-06 
       3    1.065423859    1.092537917 8.668835233e-05 
       4   0.9985319699    1.007958203 -2.912209131e-06 
       5    0.900985511   0.9988335608 5.608757991e-05 
...
     100    1.132894095    1.032154224 0.0001276646855 
\end{verbatim}
\end{bigbox}


\subsection{Using the Surfpack Executable}

This example uses Surfpack to evaluate a saved surrogate at user
provided points.  Surfpack can also generate simple designs at which
to evaluate the surrogate using the CreateSample command.  The
commands for evaluation are in {\tt sp\_eval\_surrogate.spk}:\\
\begin{bigbox}
\verbatiminput{sp_eval_surrogate.spk}
\end{bigbox}
Running {\tt surfpack sp\_eval\_surrogate.spk} loads the surface from
the file {\tt sp\_gp\_model.sps}, loads a
set of evaluation points from {\tt eval\_points.dat}:\\
\begin{bigbox}
\verbatiminput{eval_points.dat}
\end{bigbox}

and evaluates the model at them, writing {\tt sp\_surrogate\_evals.dat}:\\
\begin{bigbox}
\verbatiminput{sp_surrogate_evals.dat}
\end{bigbox}

The Surfpack phases for building and evaluating a surrogate model may
be combined into a single Surfpack command file if desired.

\subsection{Using Surfpack from a C Program}

While the C++ interface to Surfpack offers richer functionality, a
limited C interface is provided to load and evaluate a previously
saved surrogate model from a C program.

{\bf Requirements:} Linking to surfpack from C requires inclusion of {\tt
  surfpack\_c\_interface.h} (which documents the API) and linking
against (1) the surfpack libraries {\tt surfpack}, {\tt
  surfpack\_fortran}, (2) TPLs directly used by surfpack {\tt
  ncsuopt}, {\tt conmin}, and (3) system libraries {\tt
  boost\_serialization}, {\tt lapack}, {\tt blas}, and the standard
C++ libraries.  The non-system dependencies are included in binary
distributions of Surfpack and Dakota in {\tt include/} or {\tt lib/}.

{\bf Example:} The source distribution example in {\tt
  surfpack/Examples/CInterface} demonstrates loading and evaluating a
surfpack model from C.  A representative {\tt CMakeLists.txt} and {\tt
  eval\_model.c} are included.  This example can also be obtained
directly from \\
{\em
  https://software.sandia.gov/trac/surfpack/browser/trunk/examples/CInterface},
should a source distribution not be available.

\end{document}

% LocalWords:  Surfpack gaussian surfpack kriging dakota B1 LHS x1 x2 gp sps B2
% LocalWords:  freeform whitespace dat exe sp spk E1 E2 rst evals dak API TPLs
% LocalWords:  eval fortran ncsuopt conmin lapack blas CMakeLists txt
